Clustering multilingual documents by estimating text - to - text semantic relatedness

This thesis is about multilingual document clustering through estimating semantic relatedness between multilingual texts. Specifically we focus on the task of clustering multilingual documents with very limited or no supervisory information. We present two approaches to address the problem : a comparable-corpora based approach and a web-searches based approach. Our first approach derives pairwise constraints from comparable corpora and cluster multilingual documents in a semi-supervised manner. The method models document collections as weighted graph, and supervisory information is given as sets of must-link constraints for documents in different languages. Recursive k-nearest neighbor similarity propagation is used to exploit the prior knowledge and estimate semantic relatedness of multilingual documents. Spectral method is then applied to find the best cuts of the graph. Our second approach to multilingual document clustering uses web-searches to estimate semantic relatedness between multilingual words. In this approach, we extract informative terms from each language involved and query a search engine using each pair of extracted terms as keywords to construct a web-count based word similarity matrix. A variant of hierarchical agglomerative clustering algorithm is then applied to discover multilingual word clusters, and the resulting word clusters are utilized as features to perform document clustering. Evaluation of experimental results using various evaluation measures suggests that the proposed algorithms achieve satisfactory clustering result and outperforms existing methods which utilize similar supervisory information. Furthermore, since we do not use any language dependent information in the clustering process, our algorithm can be applied to documents which are written in different writing systems, such as Japanese and English texts.